Assessing the utility of common arguments used in expert review of in silico predictions as part of ICH M7 assessments.

Expert review of two predictions, made by complementary (quantitative) structure-activity relationship models, to an overall conclusion is a key component of using in silico tools to assess the mutagenic potential of impurities as part of the ICH M7 guideline. In lieu of a specified protocol, numerous publications have presented best practise guides, often indicating the occurrence of common prediction scenarios and the evidence required to resolve them. A semi-automated expert review tool has been implemented in Lhasa Limited's Nexus platform following collation of these common arguments and assignment to the associated prediction scenarios made by Derek Nexus and Sarah Nexus. Using datasets primarily donated by pharmaceutical companies, an automated analysis of the frequency these prediction scenarios occur, and the likelihood of the associated arguments assigning the correct resolution, could then be conducted. This article highlights that a relatively small number of common arguments may be used to accurately resolve many prediction scenarios to a single conclusion. The use of a standardised method of argumentation and assessment of evidence for a given impurity is proposed to improve the efficiency and consistency of expert review as part of an ICH M7 submission.

Risk assessment in the 21st century: where are we heading?

Reliance on animal tests for chemical safety assessment is increasingly being challenged, not only because of ethical reasons, but also because they procrastinate regulatory decisions and because of concerns over the transferability of results to humans. New approach methodologies (NAMs) need to be fit for purpose and new thinking is required to reconsider chemical legislation, validation of NAMs and opportunities to move away from animal tests. This article summarizes the presentations from a symposium at the 2022 Annual Congress of the British Toxicology Society on the topic of the future of chemical risk assessment in the 21st century. The symposium included three case-studies where NAMs have been used in safety assessments. The first case illustrated how read-across augmented with some in vitro tests could be used reliably to perform the risk assessment of analogues lacking data. The second case showed how specific bioactivity assays could identify an NAM point of departure (PoD) and how this could be translated through physiologically based kinetic modelling in an in vivo PoD for the risk assessment. The third case showed how adverse-outcome pathway (AOP) information, including molecular-initiating event and key events with their underlying data, established for certain chemicals could be used to produce an in silico model that is able to associate chemical features of an unstudied substance with specific AOPs or AOP networks. The manuscript presents the discussions that took place regarding the limitations and benefits of these new approaches, and what are the barriers and the opportunities for their increased use in regulatory decision making.

Development of a network of carcinogenicity adverse outcome pathways and its employment as an evidence framework for safety assessment

The traditional paradigm for safety assessment of chemicals for their carcinogenic potential to humans relies heavily on a battery of well-established genotoxicity tests, usually followed up by long-term, high-dose rodent studies. There are a variety of problems with this approach, not least that the rodent may not always be the best model to predict toxicity in humans. Consequently, new approach methodologies (NAMs) are being developed to replace or enhance predictions coming from the existing assays. However, a combination of the data arising from NAMs is likely to be required to improve upon the current paradigm, and consequently a framework is needed to combine evidence in a meaningful way. Adverse outcome pathways (AOPs) represent an ideal construct on which to organize this evidence. In this work, a data structure outlined previously was used to capture AOPs and evidence relating to carcinogenicity. Knowledge held within the predictive system Derek Nexus was extracted, built upon, and arranged into a coherent network containing 37 AOPs. 60 assays and 351 in silico alerts were then associated with KEs in this network, and it was brought to life by associating data and contextualizing evidence and predictions for over 13,400 compounds. Initial investigations into using the network to view knowledge and reason between evidence in different ways were made. Organizing knowledge and evidence in this way provides a flexible framework on which to carry out more consistent and meaningful carcinogenicity safety assessments in many different contexts.

Employing an adverse outcome pathway framework for weight-of-evidence assessment with application to the ICH S1B guidance addendum.

Across industry, there is a paradigm shift occurring for carcinogenicity testing, with the focus moving from long term animal studies to alternative approaches. Based on the explorative work done in recent years, the International Council for Harmonization (ICH) recently published a draft addendum to the S1B guidance, which allows for a weight-of-evidence (WoE) assessment to be conducted based on data gathered throughout the pharmaceutical development process and literature to mitigate some testing in rodents if the body of evidence clearly shows undertaking an animal lifetime study would not add value to the risk assessment. While several alternative approaches already exist, and other new approach methodologies (NAMs) are being explored, all of which can contribute to this WoE, it is important that all the evidence can be combined in a meaningful and consistent way to reach a conclusion. Adverse outcome pathways have been advocated as a framework for organising evidence in an integrated approach to testing and assessment, which gives context to data and can aid reaching a conclusion as to the adverse outcome (AO). This approach can be combined with a reasoning methodology to give a prediction for an AO and applied to the factors which need to be considered for the ICH S1B WoE to predict for carcinogenicity. Using this approach to the WoE assessment, consistent, scientifically robust, and transparent calls can be made as to whether conducting an animal carcinogenicity study would add value to a human risk assessment and mitigate the need to run animal studies unnecessarily.

Beyond adverse outcome pathways: making toxicity predictions from event networks, SAR models, data and knowledge.

Adverse outcome pathways have shown themselves to be useful ways of understanding and expressing knowledge about sequences of events that lead to adverse outcomes (AOs) such as toxicity. In this paper we use the building blocks of adverse outcome pathways-namely key events (KEs) and key event relationships-to construct networks which can be used to make predictions of the likelihood of AOs. The networks of KEs are augmented by data from and knowledge about assays as well as by structure activity relationship predictions linking chemical classes to the observation of KEs. These inputs are combined within a reasoning framework to produce an information-rich display of the relevant knowledge and data and predictions of AOs both in the abstract case and for individual chemicals. Illustrative examples are given for skin sensitization, reprotoxicity and non-genotoxic carcinogenicity.

Management of pharmaceutical ICH M7 (Q)SAR predictions - The impact of model updates.

Pharmaceutical applicants conduct (Q)SAR assessments on identified and theoretical impurities to predict their mutagenic potential. Two complementary models-one rule-based and one statistical-based-are used, followed by expert review. (Q)SAR models are continuously updated to improve predictions, with new versions typically released on a yearly basis. Numerous releases of (Q)SAR models will occur during the typical 6-7 years of drug development until new drug registration. Therefore, it is important to understand the impact of model updates on impurity mutagenicity predictions over time. Compounds representative of pharmaceutical impurities were analyzed with three rule- and three statistical-based models covering a 4-8 year period, with the individual time frame being dependent on when the individual models were initially made available. The largest changes in the combined outcome of two complementary models were from positive or equivocal to negative and from negative to equivocal. Importantly, the cumulative change of negative to positive predictions was small in all models (<5%) and was further reduced when complementary models were combined in a consensus fashion. We conclude that model updates of the type evaluated in this manuscript would not necessarily require re-running a (Q)SAR prediction unless there is a specific need. However, original (Q)SAR predictions should be evaluated when finalizing the commercial route of synthesis for marketing authorization.

The importance of expert review to clarify ambiguous situations for (Q)SAR predictions under ICH M7.

The use of in silico predictions for the assessment of bacterial mutagenicity under the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) M7 guideline is recommended when two complementary (quantitative) structure-activity relationship (Q)SAR models are used. Using two systems may increase the sensitivity and accuracy of predictions but also increases the need to review predictions, particularly in situations where results disagree. During the 4th ICH M7/QSAR Workshop held during the Joint Meeting of the 6th Asian Congress on Environmental Mutagens (ACEM) and the 48th Annual Meeting of the Japanese Environmental Mutagen Society (JEMS) 2019, speakers demonstrated their approaches to expert review using 20 compounds provided ahead of the workshop that were expected to yield ambiguous (Q)SAR results. Dr. Chris Barber presented a selection of the reviews carried out using Derek Nexus and Sarah Nexus provided by Lhasa Limited. On review of these compounds, common situations were recognised and are discussed in this paper along with standardised arguments that may be used for such scenarios in future.

Important considerations for the validation of QSAR models for in vitro mutagenicity.

While high-level performance metrics generated from the validation of quantitative structure-activity relationship (QSAR) systems can provide valuable information on how well these models perform and where they need to be improved, they require appropriate interpretation. There is no universal performance metric which will answer all of the questions a user might ask relating to a model, and therefore, a combination of metrics should usually be considered. Furthermore, results may vary according to the chemical space being used to validate a model, and, in some cases, it may be the validation data which is lacking or ambiguous rather than the prediction being made. Finally, users also need to consider the interpretability of the predictions being made, alongside the accuracy of the predictions. In this paper, we will discuss these important considerations in more detail within the context of the results obtained at Lhasa Limited as part of the National Institute of Health Sciences (NIHS) QSAR challenge project.

Extrapolation of in vitro structural alerts for mutagenicity to the in vivo endpoint.

As part of the hazard and risk assessment of chemicals in man, it is important to assess the ability of a chemical to induce mutations in vivo. Because of the commonalities in the molecular initiating event, mutagenicity in vitro can correlate well to the in vivo endpoint for certain compound classes; however, the difficulty lies in identifying when this correlation holds true. In silico alerts for in vitro mutagenicity may therefore be used as the basis for alerts for mutagenicity in vivo where an expert assessment is carried out to establish the relevance of the correlation. Taking this into account, a data set of publicly available transgenic rodent gene mutation assay data, provided by the National Institute of Health Sciences of Japan, was processed in the expert system Derek Nexus against the in vitro mutagenicity endpoint. The resulting predictivity was expertly reviewed to assess the validity of the observed correlations in activity and mechanism of action between the two endpoints to identify suitable in vitro alerts for extension to the in vivo endpoint. In total, 20 alerts were extended to predict in vivo mutagenicity, which has significantly improved the coverage of this endpoint in Derek Nexus against the data set provided. Updating the Derek Nexus knowledge base in this way led to an increase in sensitivity for this data set against this endpoint from 9% to 66% while maintaining a good specificity of 89%.

Improvement of quantitative structure-activity relationship (QSAR) tools for predicting Ames mutagenicity: outcomes of the Ames/QSAR International Challenge Project.

The International Conference on Harmonization (ICH) M7 guideline allows the use of in silico approaches for predicting Ames mutagenicity for the initial assessment of impurities in pharmaceuticals. This is the first international guideline that addresses the use of quantitative structure-activity relationship (QSAR) models in lieu of actual toxicological studies for human health assessment. Therefore, QSAR models for Ames mutagenicity now require higher predictive power for identifying mutagenic chemicals. To increase the predictive power of QSAR models, larger experimental datasets from reliable sources are required. The Division of Genetics and Mutagenesis, National Institute of Health Sciences (DGM/NIHS) of Japan recently established a unique proprietary Ames mutagenicity database containing 12140 new chemicals that have not been previously used for developing QSAR models. The DGM/NIHS provided this Ames database to QSAR vendors to validate and improve their QSAR tools. The Ames/QSAR International Challenge Project was initiated in 2014 with 12 QSAR vendors testing 17 QSAR tools against these compounds in three phases. We now present the final results. All tools were considerably improved by participation in this project. Most tools achieved >50% sensitivity (positive prediction among all Ames positives) and predictive power (accuracy) was as high as 80%, almost equivalent to the inter-laboratory reproducibility of Ames tests. To further increase the predictive power of QSAR tools, accumulation of additional Ames test data is required as well as re-evaluation of some previous Ames test results. Indeed, some Ames-positive or Ames-negative chemicals may have previously been incorrectly classified because of methodological weakness, resulting in false-positive or false-negative predictions by QSAR tools. These incorrect data hamper prediction and are a source of noise in the development of QSAR models. It is thus essential to establish a large benchmark database consisting only of well-validated Ames test results to build more accurate QSAR models.

Utility of published DNA reactivity alerts.

The identification of impurities with mutagenic potential is required for any potential pharmaceutical. The ICH M7 guidelines state that two complementary in silico toxicity prediction tools may be used to predict the mutagenic potential of pharmaceutical impurities. An expert review of the resulting in silico predictions is required, and numerous publications have been released to guide the expert review process. One such publication suggests that literature-based structural alerts (LBSAs) may provide a suitable aid in the expert review process. This publication provides a study of the effect of using one such set of LBSAs for the expert review of mutagenicity predictions from two complementary in silico tools. The analysis was performed using an Ames test dataset of 2619 compounds, and required interpretation of the LBSAs which proved to be a subjective process. Globally the LBSAs produced many more false positives than the in silico systems; whilst some exhibited a predictive performance comparable to the in silico systems, the majority were overly sensitive at the cost of accuracy. Use of LBSAs as part of an expert review process, without considering mitigating factors, could result in many more false positives and potentially the need to carry out additional and unnecessary Ames tests.

Using in vitro structural alerts for chromosome damage to predict in vivo activity and direct future testing.

While the in vivo genotoxicity of a compound may not always correlate well with its activity in in vitro test systems, for certain compound classes a good overlap may exist between the two endpoints. The difficulty, however, lies in establishing the cases where this relationship holds true and selecting the most appropriate protocol to highlight any potential in vivo hazard. With this in mind, a project was initiated in which existing structural alerts for in vitro chromosome damage in the expert system Derek Nexus were assessed for their relevance to in vivo activity by assessing their predictivity against an in vivo chromosome damage data set. An expert assessment was then made of selected alerts. Information regarding the findings from specific in vivo tests was added to the alert along with any significant correlations between activity and test protocol or mechanism. A total of 32 in vitro alerts were updated using this method resulting in a significant improvement in the coverage of in vivo chromosome damage in Derek Nexus against a data set compiled by the mammalian mutagenicity study group of Japan. The detailed information relating to in vivo activity and protocol added to the alerts in combination with the mechanistic information provided will prove useful in directing the further testing of compounds of interest.

Evaluation of a statistics-based Ames mutagenicity QSAR model and interpretation of the results obtained.

The relative wealth of bacterial mutagenicity data available in the public literature means that in silico quantitative/qualitative structure activity relationship (QSAR) systems can readily be built for this endpoint. A good means of evaluating the performance of such systems is to use private unpublished data sets, which generally represent a more distinct chemical space than publicly available test sets and, as a result, provide a greater challenge to the model. However, raw performance metrics should not be the only factor considered when judging this type of software since expert interpretation of the results obtained may allow for further improvements in predictivity. Enough information should be provided by a QSAR to allow the user to make general, scientifically-based arguments in order to assess and overrule predictions when necessary. With all this in mind, we sought to validate the performance of the statistics-based in vitro bacterial mutagenicity prediction system Sarah Nexus (version 1.1) against private test data sets supplied by nine different pharmaceutical companies. The results of these evaluations were then analysed in order to identify findings presented by the model which would be useful for the user to take into consideration when interpreting the results and making their final decision about the mutagenic potential of a given compound.